Systems and methods of detecting motion

ABSTRACT

Motion is detected within a defined proximity of a vehicle or fixed location equipped with a recording system by correlating frame-to-frame changes in the video streams of two or more cameras with converging views.

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit under 35 U.S.C. § 119(e) of U.S.Provisional Patent Application Ser. No. 62/376,011 filed on Aug. 17,2016, entitled “Systems and Methods of Detecting Motion,” which isincorporated by reference in its entirety.

FIELD OF THE DISCLOSURE

The present disclosure generally relates to motion detection, and moreparticularly to motion detection by correlating frame-to-frame changesin the video streams of two or more cameras with converging views.

BACKGROUND

Surveillance cameras, that is video cameras associated with recordingequipment for security purposes, are widely found in and aroundcommercial and residential buildings. These cameras are consideredextremely useful in the deterrence of crime and vandalism as well as thecapture and prosecution of culprits. Surveillance cameras associatedwith motor vehicles of all types are much less prevalent, and areprimarily so-called dash-cams, which record one or two scenes (front andrear). Many of these cameras record only when the engine of the motorvehicle is running, but some have a parking mode in which recording istriggered by motion or vibration. This mode is intended to saverecording space, since in a vehicle the recording medium is generally anSD flash card with limited capacity, and security is generally a problemduring the long periods when the vehicle is idle.

Motion detection is also used with fixed-location surveillance systemsto time-mark the points in long hours of video recording where there maybe a security interest, such as a possible intruder. A “motiondetection” function in a surveillance system commonly consists ofcomparing successive frames of a video stream and identifyingdifferences. Differences in a scene from one frame to another thatexceed a preset threshold are declared to be “motion.” This is asimplistic approach, but suitable to the limited processing poweravailable. Since the approach is susceptible to scene changes that areof no security interest (for example, pedestrians on the other side of astreet or tree limbs moving with the wind), such systems also provide ameans for the user to manually “mask out” parts of the scene to avoidfalse positives. The user may be provided with a graphical userinterface displaying the scene from a camera overlaid with a grid. Theuser then selects squares of the grid to be ignored, or masked, for thepurpose of motion detection. The scene itself is usually recorded in itsentirety. This method is reasonably effective when surveilling a fixedlocation and where it is possible to angle the camera to allow certainparts of the scene to be isolated and masked to exclude false positives.For example, if a camera can be mounted on an exterior wall and angleddownward in such as way that a driveway can be separated in its viewfrom the alley it connects to, then the alley can be masked, and motiondetection limited to changes that occur in the driveway. However, thismethod is not suitable for surveillance from a vehicle, since thevehicle can in principle be parked anywhere and it is not convenient forthe vehicle operator to manually review and mask the scene from eachcamera every time the vehicle is parked. Furthermore, it is not suitablewhere a camera view must be substantially level with the ground, suchthat background scenery registers in the same image region of the imageas a potential security interest, for example, a person moving in anarea where there is a busy street in the background.

Another well-known method of motion detection is the Passive Infra-Red(PIR) detector, which relies on infrared radiation emitted from warmobjects (such as humans) that can be distinguished from backgroundinfrared. A PIR detector is commonly used to control outdoor lights.However, it will not reliably detect people ‘bundled up’ with layers ofclothing, or vehicles that are at the same temperature as thesurroundings. Also, since its range of detection is controlled by itsgross sensitivity and the level of infrared emitted, it is not possibleto accurately define a range of proximity that it can be limited to.Therefore what is needed is a better method of detecting anddiscriminating motion that may be of interest from a security standpointfrom motion of no interest in a surveillance system.

SUMMARY

Embodiments of the present disclosure may provide means of identifyingscene changes as of interest (Scenery of Interest) by correlating intime the scene changes between two or more cameras set up to haveconverging views, optionally further correlating in time the luminosityand/or color of such changes, and using the parallax between them tocompute proximity. The correlation in combination with computedproximity within a predefined range together constitute an indication ofmotion of interest. This process will be referred to as coordinateddifferential detection (“CDD”). A scene change is defined as a change toa limited area of an image, in successive images in time or frames of avideo stream.

Embodiments of the present disclosure may provide two video camerashorizontally displaced from each other, with converging fields of view;means of detecting frame-to-frame scene changes in a camera, thatoutputs the coordinates of each area of change in the camera's image andoptionally the hue and luminosity; further means that takes the outputof the previous means for each camera and correlates scene changes thatare simultaneous between cameras, of comparable vertical height, andoptionally of comparable luminosity and/or hue; then outputs thehorizontal image coordinates; further means that evaluates thedisplacement of horizontal coordinates to determine the distance to themovement causing the scene change, and outputs a signal that may be usedto start recording if the distance is within a preset limit, or to markan ongoing recording, or to raise an alarm. The means described hereinmay be relatively easy to implement in low-cost commercial programmablelogic, compared with much more sophisticated and elaborate imageprocessing needed to identify and track objects using softwarealgorithms, and then use that information to estimate distance andinterest. In some cases it may be necessary displace the camerasvertically instead of horizontally. Then the parallax is determined bythe displacement of vertical coordinates in the camera images, ratherthan horizontal. In general, the geometry of the system may be rotatedaround an axis from the cameras to the area under surveillance withoutchanging the operating principle.

Some embodiments of the present disclosure may provide a method ofidentifying scene changes as of interest, the method comprising:detecting frame-to-frame scene changes in two video cameras displacedfrom one another and having converging fields of view to outputcoordinates of each area of change; using the coordinates of each areaof change, correlating scene changes that are simultaneous between thetwo cameras of comparable vertical height, and outputting imagecoordinates; evaluating displacement of the image coordinates todetermine distance to movement causing the scene changes, and outputtinga signal that instructs performance of one of a plurality of actions.The method also may comprise correlating in time color of the scenechanges; and computing proximity using a parallax between the scenechanges. The method may further comprise correlating in time luminosityof the scene changes; and computing proximity using a parallax betweenthe scene changes. The scene changes may be changes to a limited area ofan image in successive images in time or frames of a video stream.

Further embodiments of the present disclosure may provide a system forsurveilling an area around a motor vehicle, the system comprising: oneor more camera pods attached to a roof of the motor vehicle; and acentral recording unit connected to the one or more camera pods, whereinthe central recording unit supplies power to the one or more camerapods, sends instructions to the one or more camera pods, receivesstreaming video and/or successive still images from the one or morecamera pods, performs coordinated differential detection, and records toits own data storage or a remote data storage. One or more flat powerand data cables may connect the one or more camera pods to the centralrecording unit. The central recording unit may be mounted under a driverseat of the motor vehicle. Each of the one or more camera pods mayinclude two cameras having fields of view pointing in oppositedirections along an axis inclined at approximately 45 degrees to an axisof the motor vehicle, wherein the fields of view may create a sector ofconvergence where an object appears to a pair of cameras. The sector ofconvergence may be a primary sector of convergence when created bycameras in adjacent ones of the one or more camera pods. The sector ofconvergence may be a secondary sector of convergence when created bynon-adjacent cameras. Each of the two cameras may include a lensassembly, a lens holder, and an image sensor. Each of the one or morecamera pods may include a baseboard having a field programmable gatearray (FPGA), wherein the FPGA may communicate with the centralrecording unit and control each camera's mode of operation and whetherit captures streaming video or successive still images. The system mayhave a drive mode of operation and a park mode of operation. When thesystem is in a drive mode of operation, the central recording unit mayreceive streaming video data from each camera and the streaming videodata may be recorded onto the data storage. When the system is in a parkmode of operation, the system may run on battery power supplied by themotor vehicle and images may be sampled using coordinated differentialdetection.

Additional embodiments of the present disclosure may provide a systemfor surveilling an area around a motor vehicle, the system comprising:one or more camera pods attached to a roof of the motor vehicle, each ofthe one or more camera pods including two cameras having fields of viewpointing in opposite directions along an axis inclined at approximately45 degrees to an axis of the motor vehicle, wherein the fields of viewmay create a sector of convergence where an object appears to a pair ofcameras; and a central recording unit connected to the one or morecamera pods and including a field programmable gate array (FPGA) orother low-power logic device that performs coordinated differentialdetection for the sector of convergence to identify an approximateposition for the object in the area around the motor vehicle. The FPGAmay be configured to capture image frames from each of the two camerasat a predetermined interval, decompress image data associated with theimage frames to create a decompressed frame, and compute a differentialwith a stored decompressed frame. The FPGA may be further configured toidentify differentials for selected pairs of image frames that share anapproximate height above ground and involve comparable color hues orluminosities. The central recording unit may perform a sequence ofcoordinated differential detection to approximate trajectory and speedof the object in the area around the motor vehicle. The FPGA may beconfigured to perform a decimation of differentials derived from each ofthe cameras to arrive at a lower resolution for use in correlation. TheFPGA may be configured to correlate differentials at approximately thesame vertical height between the two cameras.

Other technical features may be readily apparent to one skilled in theart from the following figures, descriptions and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of this disclosure, reference is nowmade to the following description, taken in conjunction with theaccompanying drawings, in which:

FIG. 1 depicts an illustration looking down at the top of arepresentative motor vehicle, depicting the locations of pods containingpairs of video cameras arranged to acquire scenery around the motorvehicle according to an embodiment of the present disclosure;

FIG. 2a depicts the fields of view of eight cameras disposed in fourpods according to an embodiment of the present disclosure;

FIG. 2b illustrates primary and secondary converging fields of viewaccording to an embodiment of the present disclosure;

FIG. 2c illustrates diverging areas of convergence created by rotatingthe cameras slightly according to an embodiment of the presentdisclosure;

FIG. 3a illustrates the configuration of cameras and printed circuitboards inside a pod in plan view according to an embodiment of thepresent disclosure;

FIG. 3b illustrates an elevation of the cameras and printed circuitboards inside a pod according to an embodiment of the presentdisclosure;

FIG. 3c is a plan view with the cameras reoriented slightly in order toachieve the fields of view illustrated in FIG. 2c according to anembodiment of the present disclosure;

FIG. 4 depicts a form of flat power and data cables that attach to theexterior of a motor vehicle for wiring pods to a Central Recording Unitaccording to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram illustrating how power and data signalsmay be transferred into and through a camera pod according to anembodiment of the present disclosure;

FIG. 6a is a schematic diagram of a Central Recording Unit depicting theconnections to the camera pods as well as the connection to the motorvehicle battery according to an embodiment of the present disclosure;

FIG. 6b is a schematic diagram of a Central Recording Unit showing theFPGA responsible for coordinated differential detection and thecomponents needed for recording according to an embodiment of thepresent disclosure;

FIG. 7a depicts the regions in a sector of convergence defined by alook-up table of coarse horizontal components according to an embodimentof the present disclosure;

FIG. 7b illustrates identifying the trajectory of a moving object bymeans of successive coordinated differential detection according to anembodiment of the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure apply the principle of coordinateddifferential detection (CDD) to the problem of surveillance around amotor vehicle, such as a passenger car, and in particular, the problemof maintaining continuous surveillance during extended periods when themotor vehicle is parked and the engine switched off. Even with thestorage technologies available today, continuous video recording frommultiple points of view for many hours can require infeasibly largestorage capacity. Furthermore, continuous streaming and recording canrequire more electrical energy than the motor vehicle's battery cansupply for extended periods, without compromising its ability to startthe engine. Embodiments of the present disclosure further illustrate theprinciple applied as an after-market product attached to the motorvehicle, although it will be appreciated that virtually the sameembodiment may be built into the structure of the motor vehicle by itsoriginal manufacturer.

Referring to FIG. 1, camera pods 101 may be attached at four locationson top of the roof of passenger car 100. Flat power and data cables 102may interconnect pods 101 with Central Recording Unit 103, mounted underthe driver's seat, which may supply power to pods 101, send instructionsto pods 101, receive streaming video and/or successive still images frompods 101, perform CDD, and record to its own data storage. CentralRecording Unit 103 will be discussed in more detail herein.

Referring to FIG. 2a , pod 101 may contain two cameras, along with othersupporting electronics which will be described herein. In thisillustration, each pod 101 may contain cameras with fields of viewpointing in opposite directions along an axis inclined at 45 degrees tothe axis of the motor vehicle, for example, fields of view 202 and 203.The fields of view may be assumed to subtend approximately 90 degrees,as this is a value that is easily attainable with inexpensive camerasand lenses. The particular fields of view 201 through 204 will bedescribed in the following figures.

Referring to FIG. 2b , fields of view 201 and 202 may create sector ofconvergence 210, where an object would appear to both cameras associatedwith these fields of view. Sectors of convergence 210 and 211 may becalled primary sectors of convergence because they are created bycameras in adjacent pods. In contrast, there may be secondary sectors ofconvergence, such as 212, created by non-adjacent cameras, in this case,those associated with fields of view 201 and 204. Preferably, the axesof the fields of view may be rotated slightly from an exact 45 degreeincline in order to improve the shapes of the sectors of convergence.Referring to FIG. 2c , sectors 220, 221, and 222 may correspond tosectors 210, 211, and 212 of FIG. 2b , but a slight rotation of eachcamera axis may make the distribution of the sectors around the motorvehicle more uniform.

Consider object 225, which is on two lines of sight 226 in fields ofview 201 and 202. If object 225 is in motion, changes in scenery mayoccur simultaneously along both lines of sight at comparable heightsfrom the ground, signaling movement within sector 220. The fact ofsimultaneity signals the presence of object 225 without thecomputationally expensive image processing needed for actual objectrecognition. The angles of the lines of sight may determine the positionof object 225, including its distance from motor vehicle 100. Thus,motion in a defined proximity of the vehicle may be detected, as will beexplained below.

Continuing with FIG. 2c , there may be sectors such as 223 and 224 thatexist in only one field of view, which is a consequence of assuming theindividual fields of view subtend no more than 90 degrees. Considerobject 227 in sector 224, appearing only in field of view 204. Here, thenarrow shape of sector 224 may be exploited to determine the position ofobject 227, the presence of which is signaled by changes in field ofview 204 that are not simultaneous with corresponding changes in fieldof view 203 or field of view 201. Object 227 can only be localized tosomewhere along line of sight 228 that is not within sectors 221 or 222,but due to the narrowness of sector 224, this is acceptable.

Referring to FIG. 3a , this is a plan view of the assembly inside camerapod 101, and FIG. 3b is an elevation view of same. The assembly mayinclude baseboard 300, which is a printed circuit board with a FieldProgrammable Gate Array (FPGA 500, below) and supporting circuitry,which will be discussed further below. Each camera may include lensassembly 301, lens holder 302, and an image sensor which may be coveredby lens holder 302 and soldered to daughterboard 303. Right-angleheaders 305 may serve to connect signals and power from baseboard 300 toeach daughterboard 303. There may be two I/O connectors 304 soldered tobaseboard 300 which may carry power and signals from flat power and datacables 102.

FIG. 3a and FIG. 3b depict the cameras as aimed in opposing directions,which may give the fields of view illustrated in FIG. 2a and FIG. 2b .In order to achieve the more optimal fields of view illustrated in FIG.2c , the upper camera in FIG. 3a may be reoriented 10 degrees clockwise,while the lower camera may be reoriented 10 degrees counter-clockwise,as depicted in FIG. 3c . This configuration may be used in all fourlocations of camera pod 101.

FIG. 4, depicting the form of flat power and data cables 102 accordingto an embodiment of the present disclosure. Two conductor pairs 402 and403 may be embedded in a weatherproof insulating tape that isadhesive-backed for attaching to the exterior of motor vehicle 100. Eachconductor pair may carry signals to and from a pod using differentialsignaling. The two pairs together also may carry power to each pod usingthe well-known “phantom pair” technique, used, for example, inpower-over-Ethernet schemes.

FIG. 5 depicts schematically the components of a camera pod and the flowof signals and power according to an embodiment of the presentdisclosure. Flat power and data cables 102 may plug into connectors 304mounted on baseboard 300. If this pod is closest to Central RecordingUnit (CRU) 103, then the cable plugged into J1 may connect to CRU 103,and the cable that plugs into J2 may connect the next pod on the sameside of the motor vehicle. (See FIG. 1.) Otherwise, the cable pluggedinto J1 may connect to the pod closest to CRU 103 on the same side ofthe car, and nothing may plug into J2.

There may be two dual pulse transformers 504. T1B may provide electricalisolation for a differential signal on the first conductor pair from J1to FPGA 500. T1A may be cross-connected to T2B. If this pod is closestto CRU 103, the differential signal on the second pair from J1 may be onthe first pair at the next pod, due to this cross-connection, althoughthere may be a small amount of insertion loss due to T1A and T2B. T2Amay carry no differential signal, and R1 may only be used to dampen anyelectrical noise. Thus, the same camera pod design may be used in eachlocation.

Pulse transformers 504 may block the flow of direct current, which mayprevent power from being applied to the signal pins of FPGA 500.Instead, power may flow longitudinally through each cable pair from CRU103 to each pod. Approximately +18V may flow from right to left in theupper pair of each cable 102, referenced in the orientation of FIG. 5.Likewise, the return current path may be from left to right in the lowerpair of each cable, at approximately the same voltage potential as thechassis of the motor vehicle. Each camera pod may be electricallyinsulated from the motor vehicle.

Power may flow through each transformer in the common mode, generatingno net magnetic flux to affect differential signals, and may be accessedusing the center taps. Capacitors 503 may provide noise decoupling.Current may pass from right to left through connection 502 and left toright through the pod ground reference. Power may be regulated forinternal use by step-down regulator 501. Regulated power may bedistributed to the FPGA and to camera connectors 506 over power bus 505.CRU 103 may provide solid state current limiting as well as fuseprotection for the power delivered over cables 102, so that damage to acable or pod may not create a hazard.

FPGA 500 may communicate with CRU 103 using low voltage differentialsignaling in a half-duplex poll/response manner. Signals may be sent aspackets of varying lengths, e.g. short packets for commands from theCRU, and long packets for frames of video data. FPGA 500 may communicateto cameras 507 through connectors 506. Each camera may comprise 301,302, 303, and 305 of FIGS. 3a and 3b . FPGA 500 may control eachcamera's mode of operation and whether it captures video frames or stillimages. In this embodiment of the present disclosure, FPGA 500 maycompress each image or video frame using JPEG compression, which mayprovide an optimal trade-off between compression level and computingpower needed. JPEG compression of video (Motion JPEG) also may allow forrapid random access of individual frames in video files or streams bythe CRU, again with limited computing power. More advanced compression,such as MPEG, may require decompressing larger chunks of frames to getat one.

In this embodiment of the present disclosure, the bit rate transmittedover each differential pair may be 62 Mbs. This may allow video framescompressed to 125 kB per frame, or 1 Mbit, to be sent back to the CRU atup to 30 frames per second, while leaving idle time for receivingcommands. The packets may not be formatted as true Ethernet packets, asthere is no need for a MAC address. The packet overhead may be limitedto a preamble for syncing, a packet type indicator, and a frame checksequence (cyclic redundancy check).

Central Recording Unit 103 according to an embodiment of the presentdisclosure is shown schematically in FIGS. 6a and 6b . Referring firstto FIG. 6a , CRU Baseboard 600 may be a printed circuit board thatcarries the electrical components. Cable coupling circuits 601 mayconnect to flat power and data cables 102. Four camera pods 101 may bearranged as pairs on each side of the motor vehicle. Each pair of podsmay be interconnected with a flat power and data cable between them,with another from the front pod to the CRU. (See FIG. 1.) Therefore,there may be two flat power and data cables connecting to the CRU, oneleading to the two pods on the left side of the car, and the other tothe two pods on the right side. In turn, one cable coupling circuit 601deals with the left side of the car, and the other with the right.

Within each cable coupling circuit, dual pulse transformer 602 mayprovide electrical isolation for the differential pairs, of which theremay be two, one for the front pod and one for the rear. With two cablecoupling circuits, there may be a total of four differential pairs 610,one corresponding to each camera pod. In addition, the two sides of eachpulse transformer may feed 18V power to the pods through the flat powerand data cable, in the “phantom pair” manner discussed with respect toFIG. 5. Protection circuit 603 may comprise a solid state currentlimiter employing a MOSFET pass element, both for low “on” resistancefor normal operation and for rapid shut-off in the event of anover-current, as well as a fuse for secondary protection in the event ofhazardous damage to the limiter itself. Regulator 607 may be anon-isolated boost converter that steps up the battery voltage to 18Vfor feeding power to the camera pods. The flat power and data cables mayplug into connectors 604.

A cable from the fuse box in the motor vehicle's engine compartment mayplug in to connector 605. This cable may contain three conductors.BATTERY may supply power from the motor vehicle battery and is “alwayson,” that is, not interrupted by the ignition switch. In contrast,IGNITION may supply voltage only when the ignition switch is in the “on”position. This voltage may be divided and filtered by circuit 606,allowing the CRU to sense the state of the ignition switch. Finally,CHASSIS comes from the car chassis and may serve as a ground referenceand battery return. Regulator 608 may be a non-isolated step-downconverter that supplies various voltage rails to the rest of the CRU.

Referring now to FIG. 6b , differential pairs 610 may be compatible withlow voltage differential signaling and connect to FPGA 612, which inturn may control the camera pods with commands and receives video and/orimage data compressed with JPEG, as discussed with respect to FIG. 5.Further, FPGA 612 may perform coordinated differential detection onselected pairs of cameras. Further, ignition sense 611 also may connectto FPGA 612, informing the control logic contained within whether themotor vehicle's engine is running.

FPGA 612 may communicate with, and control, single board computer (SBC)613 by means of interface 615, comprising a PCIE bus as well as signalsto put the SBC into a low-power sleep mode, wake it up, and perform ahard reset if needed. A commercial SBC running the Linux operatingsystem may be used in this embodiment of the present disclosure, but anequivalent set of integrated circuits can be soldered directly to CRUBaseboard 600 to save cost without departing from the presentdisclosure.

SBC 613 may communicate with solid state drive (SSD) 614 using interface616, which may be a standard SATA interface for computer storage.Commercial SSDs of 480 GB or 960 GB may be used in embodiments of thepresent disclosure. It will be understood that each of these blocks mayreceive power from regulator 608, which may deliver all the voltagerails necessary. SBC 613 may be equipped with various useful interfaces,such as USB for keyboard, mouse, and external backup storage, as well asWi-Fi for access by a laptop computer, tablet, or smartphone.

In summary, from an operational perspective, embodiments of the presentdisclosure may comprise eight cameras located in four camera pods 101,communicating with FPGA 612 located in CRU 103. FPGA 612 may performcoordinated differential detection, may communicate with SBC 613, whichin turn may read and write files on SSD 614, and also may sendrecordings to a remote data storage.

MODES OF OPERATION

There may be two main modes of operation depending upon the state ofignition sense 611. In the driving mode of operation, all eight camerasmay operate in full-motion video mode, with frame rates optionallyselectable up to 30 frames per second. FPGA 612 may receive streamingvideo data from each camera in Motion JPEG format and pass this data toSBC 613, which in turn may record the data onto SSD 614. In the parkmode of operation, the motor vehicle's engine may be turned off and thesurveillance system may run on the motor vehicle's battery, so the powerdraw must be minimized. Also, the motor vehicle may be parked for manymore hours than it is driven, and it may not be economical to providestorage to record full video from multiple cameras for more than a fewhours, so instead images may be sampled at a much lower frame rate andCDD may be used to determine when to record full video of scenery ofinterest (i.e., scenery identified as having movement of securityinterest).

What follows is a set of illustrative calculations that apply toembodiments of the present disclosure. It should be understood that thecalculations can vary with different video frame rates, JPEG compressionlevels, etc., in order to optimize for a particular application or theuser's desires. As in conventional surveillance systems, some usersprefer to sacrifice image resolution for greater storage, while othersmay accept lower video frame rates in order to gain higher resolutionwithout sacrificing too much storage capacity, etc.

Images from each camera 507 may be compressed with JPEG to approximately125 KB per frame. In drive mode, the streaming data from eight camerasat 24 frames per second may total up to 24 MB per second, or 86.4 GB perhour.

In park mode, imagery may be sampled at 1 frame per second, which may beaccomplished with cameras 507 programmed to operate in a “snap-shotmode” rather than full motion video, for reduced power consumption. Thedata from eight cameras may total up to 1 MB per second, or 3.6 GB perhour. To further reduce power consumption in park mode, FPGA 612 may putSBC 613 into a low-power “sleep” mode, similar to the sleep mode of alaptop computer, and wake it briefly at five-minute intervals to recordbatches of collected image data onto SSD 614.

It should be appreciated that while park mode may be considered alow-power operating mode, days of parking can still discharge the motorvehicle's battery. Therefore, the battery voltage may be comparedagainst a low-voltage threshold, and the system may be shut downcompletely if the threshold is crossed. This circuit is not shown in thedrawings, but uses well-known methods for comparing voltages andcontrolling a MOSFET pass transistor.

Storage Calculations

If a motor vehicle is being driven an hour per day, e.g., commuting toand from work, then the total data recorded may be 86.4 GB/hour×1hour+3.6 GB/hour×23 hours, or 169 GB per day. A number of options areavailable for solid state drive 614, including two or more drives foradded capacity, but a reasonably economical option is a single 480 GBdrive, which can store a little less than three day's worth as justcalculated.

SSD 614 may provide storage local to the motor vehicle. In thisembodiment of the present disclosure, so-called cloud storage also maybe available through data transmission over common 4G/LTE cell phonenetworks, or Wi-Fi hotspots where available. Cloud storage is nearlyunlimited, but the storage and especially the data transmission do costmoney, and so still need to be used efficiently. Embodiments of thepresent disclosure may use the following scheme, along with CDD, to makeefficient use of storage.

A portion of SSD 614 may be set aside for endless circular storage,meaning that new data may be appended in the storage area until the endof the area is reached. Then the process may move to the beginning ofthe area and continue appending as before. In this way, a sequentialrecord of the previous one or two days' worth of imagery may bemaintained, with older data discarded.

A second portion of SSD 614 may be set aside for extracted circularstorage. This may be operated in the same way as the endless circularstorage just discussed, but may be used to store scenery of interestidentified with CDD.

Both endless circular and extracted circular storage may be managed bySBC 613 without intervention by a user. A third portion of SSD 614 maybe set aside for permanent storage, and may be used to store scenery ofinterest marked by a user for preservation. The user can also designatethat the previous few minutes of current video or sampled imagery bepreserved as user-designated scenery in permanent storage in someembodiments of the present disclosure.

Cloud storage is an optional way of backing up scenery of interest aswell as user-designated scenery, taking advantage of the greatercapacity of cloud storage at reasonable cost, as well as minimizing thedata fees associated with transmission over a 4G/LTE network.

Coordinated Differential Detection

FPGA 612 may perform coordinated differential detection for each of theprimary and secondary sectors of convergence. At a periodic rate, suchas once per second in park mode, the FPGA may capture an image framefrom each camera and decompress the image data. For each decompressedframe, it may compute a differential with a stored previous decompressedframe, then discard the previous frame. Next, for selected pairs offrames, the FPGA may identify differentials that share a commonapproximate height above ground and involve comparable color hues, ifdaylight, or luminosities, if night time. Each correlation may be takento signal the movement of an object, although there may be no attempt torecognize the object.

For each correlation, an approximate position with respect to the motorvehicle may be determined by means of previously-calculated lookuptables. The horizontal axis of each image may be divided into 32segments, or coarse coordinates. The vertical axis of each image may besimilarly divided into 32 rows, which may be used with coordinateddifferential detection to determine that differences in the imagesframes of two cameras occur at comparable height. For each pair offrames corresponding to a sector of convergence, there may be a lookuptable with 1024 regions defined in that sector, each regioncorresponding to a pair of coarse coordinates. FIG. 7a illustrates thismethod using a smaller number of coarse coordinates for clarity. Rays 70(solid lines) may divide field of view 1 into sectors representing thecoarse horizontal coordinates in its camera's image. Similarly, rays 702(dashed lines) do the same for field of view 2. Sector of convergence220 may be divided into quadrilateral regions, where each region maycorrespond to a unique pair of coarse horizontal components. It will beapparent that there is more positional resolution closer to the motorvehicle than further away, which is perfectly suitable for theapplication.

Recall the discussion of how object 227 may be located when it appearsin only one field of view. In this case, it is the absence of CDD thatmay place object 227 within sector 224 and along line of sight 228,which may be found using a look-up table relating the single coarsecoordinate value to one of 32 regions of sector 224.

It should be appreciated that the approximate trajectory and speed of amoving object may be computed from a sequence of related CDDs, asillustrated by the partially- and fully-shaded quadrilaterals in FIG. 7b. It should be understood that the actual resolution of shadedquadrilaterals is significantly finer, since the illustration depictsfewer coarse coordinates for clarity. For example, this might be themoving backup light of another motor vehicle. Even though the system maynot have the computational power to recognize a “motor vehicle” object,it may be easily able to determine that an object is on a collisioncourse. This can be used to trigger full video from this pair of camerasin park mode, so that a full-motion video of the scene leading up to anactual collision or near-collision may be recorded, should CDD thendetermine the threshold of proximity has been crossed, as indicated bythe set of fully shaded quadrilaterals. If the object does not reach thethreshold, the video may be discarded and the cameras returned to parkmode.

In outdoor environments, especially at night, there may be stationaryobjects in proximity to a pair of cameras, such as a post or a parkedvehicle, one or more of which may be periodically illuminated by aflashing light somewhere nearby. The flash of the flashing light maycome at regular intervals, as in the case of a traffic light, or atirregular intervals, as with passing headlights. In either case, CDD maydetect what appears to be movement within the defined proximity.

In these cases, when the object is stationary, the coordinates reportedby CDD (horizontal and vertical, or range and position) may beapproximately the same with each flash of light. SBC 613 may use thisfact to identify stationary objects. The SBC may make recordings for thefirst few flashes, and subsequently may ignore them unless thecoordinates change, indicating actual movement.

Many objects of security interest, such as other vehicles, may havelarge areas of substantially similar coloration. If the object ismoving, only the edges of these areas that are roughly perpendicular tomovement (or expansion and contraction if the object is moving toward oraway) may create differentials. However, variations in coloration orshadows may also appear as edges and create differentials. In many rowsof each image frame (left and right views), there may be multipleplausible correlations. Finding the parallax for each possiblecombination may lead to a cluster of ranges around the actual range ofthe object. It may be acceptable to use (for proximity purposes) theminimum range in the cluster, or the average. It also may be possible toexclude rows where there are multiple plausible correlations, and onlyuse for proximity rows where unique correlations are found. Since theobjective is to detect motion within a defined proximity, and not toidentify the object, this may be advantageous.

Each image frame may be composed of a large number of pixels, forexample, 337,920 pixels for an NTSC-compatible frame, or 2,073,600 foran HD frame. Such resolutions may be useful for review and inspectiontasks, such as recognizing faces or reading license plates from recordedvideo, but can make identifying differentials at a common verticalheight more difficult, due to slight misalignment between the elevationof each camera's line of sight, or due to the field-of-view curvaturecaused by the relatively short focal length of a camera lens that givesa 90° field of view.

Field-of-view curvature can be partially compensated by remapping pixelsaccording to a pre-computed pattern. In addition, FPGA 612 may perform adecimation of the differentials derived from each camera, to arrive at alower resolution for use in correlation. For example, it may average theinformation in rectangles of N-by-M adjacent pixels, converting theframe image differentials into X-by-Y tiles, and then performcorrelation on the tiles instead of the original differentials. Sinceeach tile is an average of differentials from N-by-M pixels, the FPGAmay correlate differentials at approximately the same vertical heightbetween two cameras despite optical and mechanical misalignments. Theresulting horizontal coordinates may be more coarse than the originalresolution, by the factor N, but this is still consistent with theexamples previously discussed.

The optimal values for N and M, and therefore the size of the tiles, maybe determined approximately as follows. Larger tiles (N and M larger)reduce the needed capacity of FPGA 612 and allow for largermisalignments, but at the cost that more differentials, and therefore,more image frame information, may be blended together into each tile,making it more difficult to find unique correlations between cameras. Onthe other hand, if the tiles are too small (N and M too small), theeffects of misalignments also may be more pronounced, and once again, itmay become more difficult to find unique correlations between cameras.An optimal range of values may therefore be found by taking intoconsideration the expected misalignments in a manufactured product.

It should be appreciated that these techniques may accomplish thedetection of motion within a defined proximity without the need for theexpensive and battery-draining computational power that would be neededto identify and track objects in real time by means of softwarealgorithms. It should be further appreciated that, for the purposes ofcontrolling the recording and storage of scenery of interest that willbe later interpreted by human beings, this is sufficient.

User Interface

In embodiments of the present disclosure, SBC 613 may be equipped withWi-Fi, which may allow ready networking with a smart phone or laptopcomputer. An Internet browser may be used to access web pages served bySBC 613 to review and download video, and to manage the system, forexample, to designate scenery of interest for permanent storage asdiscussed above, or to change a proximity threshold for CDD.

Although the present disclosure and its advantages have been describedin detail, it should be understood that various changes, substitutionsand alterations can be made herein without departing from the spirit andscope of the disclosure as defined by the appended claims. Moreover, thescope of the present application is not intended to be limited to theparticular embodiments of the process, machine, manufacture, compositionof matter, means, methods and steps described in the specification. Forexample, the cameras may be built into the structure of the vehiclerather than mounted on its roof. As another example, some advantages ofthis system, such as low-power operation, may be applied to surveillancearound a fixed location where extended battery operation is desirabledue to possible power outages. As another example, one or more FPGAs maybe replaced by suitably low-powered application specific integratedcircuits (ASICs). As one of ordinary skill in the art will readilyappreciate from the disclosure, processes, machines, manufacture,compositions of matter, means, methods, or steps, presently existing orlater to be developed that perform substantially the same function orachieve substantially the same result as the corresponding embodimentsdescribed herein may be utilized according to the present disclosure.Accordingly, the appended claims are intended to include within theirscope such processes, machines, manufacture, compositions of matter,means, methods, or steps.

The invention claimed is:
 1. A method of identifying scene changes as ofinterest, the method comprising: detecting frame-to-frame scene changesin two video cameras displaced from one another and having convergingfields of view to output coordinates of each area of change; using thecoordinates of each area of change, correlating scene changes ofcomparable vertical height relative to the ground that are simultaneousbetween the two cameras, and outputting image coordinates; evaluatingdisplacement of the image coordinates to determine distance to movementcausing the scene changes; comparing the distance to movement causingthe scene changes to a proximity threshold; and outputting a signal thatinstructs performance of one of a plurality of actions.
 2. The method ofclaim 1 further comprising: correlating in time color of the scenechanges; and computing proximity using a parallax between the scenechanges.
 3. The method of claim 1 further comprising: correlating intime luminosity of the scene changes; and computing proximity using aparallax between the scene changes.
 4. The method of claim 1, whereinthe scene changes are changes to a limited area of an image insuccessive images in time or frames of a video stream where a stationaryflashing object has not been detected.